High Performance Multiview Video Coding

Following the standardization of the latest video coding standard High Efficiency Video Coding in 2013, in 2014, multiview extension of HEVC (MV-HEVC) was published and brought significantly better compression performance of around 50% for multiview and 3D videos compared to multiple independent single-view HEVC coding. However, the extremely high computational complexity of MV-HEVC demands significant optimization of the encoder. To tackle this problem, this work investigates the possibilities of using modern parallel computing platforms and tools such as single-instruction-multiple-data (SIMD) instructions, multi-core CPU, massively parallel GPU, and computer cluster to significantly enhance the MVC encoder performance. The aforementioned computing tools have very different computing characteristics and misuse of the tools may result in poor performance improvement and sometimes even reduction. To achieve the best possible encoding performance from modern computing tools, different levels of parallelism inside a typical MVC encoder are identified and analyzed. Novel optimization techniques at various levels of abstraction are proposed, non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) in prediction unit (PU), fractional and bi-directional ME/DE acceleration through SIMD, quantization parameter (QP)-based early termination for coding tree unit (CTU), optimized resource-scheduled wave-front parallel processing for CTU, and workload balanced, cluster-based multiple-view parallel are proposed. The result shows proposed parallel optimization techniques, with insignificant loss to coding efficiency, significantly improves the execution time performance. This , in turn, proves modern parallel computing platforms, with appropriate platform-specific algorithm design, are valuable tools for improving the performance of computationally intensive applications

Massively efficient motion estimation by exploiting inter-pixel similarities

In this paper, we propose a novel massively parallel motion estimation for HEVC using an efficient yet flexible predicate algorithm, ScalingFast that exploits inter-pixel similarities. When evaluated over four video test video sequences with changing resolution, the proposed algorithm outperforms the anchor algorithm by factor of up to 4.6, with little to no loss in rate-distortion (RD) performance

Multi-level complexity reduction for HEVC multiview coding

Standardized in 2014, multiview extension of high efficiency video coding (MV-HEVC) offers significantly better compression performance of up to 50% for multiview and 3D videos compared to multiple independent single view HEVC coding. However, the extreme high computational complexity of MV-HEVC demands significant optimization of the encoder. In this work, we propose a series of optimization techniques at various levels of abstraction: non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) for prediction units, fractional and bidirectional ME/DE, quantization parameter-based early termination of coding tree unit (CTU), and optimized resource-scheduled wave front parallel processing for CTU. When evaluated over three views for all available official multiview video coding test sequences, proposed optimization outperforms the anchor encoder by average factor of 5.4 at the cost of 4.4% bitrate (DBR) increase at no loss in PSNR, or alternatively a PSNR degradation of 0.12 dB at no change to the DBR

GPU accelerated motion and disparity estimations for multiview coding

Multiview video coding (MVC) is an extension of H.264/AVC standard. Its purpose is to further increase compression efficiency when multiple video sources are presented for coding. The computational complexity of MVC, however, is extremely high due to the motion estimation (ME) between the frames and disparity estimation (DE) between the views, contributing to more than 99% of overall complexity of the coder. For MVC to find a common place for real-time applications requires acceleration of ME and DE engines. On the other hand, there is a recent uptake of graphical processing unit (GPU) computing to significantly enhance the performance of signal and image processing applications, through massive-parallel processing. This paper presents the development and GPU implementation of a parallel full search algorithm to significantly reduce the computational complexity of ME and DE over the full search and TZsearch estimations on a sequential processor, by a factor of 300 and 4, respectively. © 2013 IEEE

Opportunities for high-level parallelism in multiview video coding

This paper investigates the problem of MVC from a high-level perspective and proposes a multiple-view-frame-parallel (MVFP), scheduling scheme for a computing cluster. The proposed scheme outperforms the view sequential coding by a factor of more than 12, for the IPP prediction structure for MVC

Decision zone-based prallel fast motion and disparity estimation scheme for multiview coding

The challenge to H.265/HEVC multiview video coding (MVC) is its extremely high computational complexity. This paper presents the development and implementation of a scalable massively parallel fast search algorithm to significantly reduce the computational cost of motion and disparity estimation over the current best available GPU full block matching and sub optimal CPU fast search algorithms by a factor up to 2.4 and 8.4, respectively

A scalable massively parallel motion and disparity estimation scheme for multiview video coding

Multiview video coding (MVC) has recently received considerable attention. It is proposed as an extension of H.264/Advanced Video Coding (AVC) standard for multiple video source compression. To resolve the extremely high computational complexity of MVC (and in fact other AVC techniques), suitable parallel algorithms need to be developed that are amenable to implementation on low-cost massively parallel architecture, platforms that have found a common place due to recent advances in the parallel computer architecture. The high complexity of MVC is due to its prediction structure, where motion estimation (ME) between the frames and disparity estimation (DE) between the views contribute to more than 99% of overall complexity of the coder. This paper presents the development and implementation of a scalable massively parallel fast search algorithm to significantly reduce the computational cost of ME/DE over the current best available full block matching, and suboptimal fast search algorithms. The proposed massively parallel fast search algorithm (DZfast), when evaluated over eight views, outperforms the existing full search and fast search MVC algorithms by a factor of up to 245.8 and 8.4, respectively. This speedup comes at no or minute loss in rate-distortion performance

Parallel multiview video coding exploiting group of pictures level parallelism

This paper presents the use of a computer cluster with heterogeneous computing components to provide concurrency and multi-level parallelism at coarse grain and massive fine-grain for multiview video coding (MVC) applications. MVC involves coding of multiple video sequences that are taken from the same scene but different perspective. In addition to motion estimation (ME) used in conventional video coding for single view video for exploiting inter-frame temporal similarities, MVC adopts disparity estimation (DE) to further increase compression. To overcome the huge computational cost associated with ME and by extension with DE, attention has been mainly focused on developing fast ME/DE algorithms. Although fast ME/DE algorithms bring substantial speedup, to achieve realtime MVC encoding, it requires further acceleration of the coding process at higher levels. Towards this end, this paper proposes a multiple-view-parallel, multiple-interleaved group of pictures (multiple-IGOP) scheduling scheme for MVC. When evaluated over eight views, with no loss in rate distortion (RD) performance, the proposed scheme outperforms view-sequential coding by a factor of up to 12.4 and 12.3, respectively, for two popular prediction structures, IBP and IPP